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Interactive simulation including force feedback. The system includes a computer game display unit programmed with simulation 
rules. A control unit is connected to the computer and provides information and control signals to the computer to interact with the stored 
simulation rules and to receive information and control signals from the computer. A movable structure for movement in at least two 
degrees-of-freedom by an operator is interconnected to the control unit by means of actuators. A programmable unit processes signals and 
information from the computer to generate signals to drive the actuators so as to apply forces in the at least two degrees-of-freedom to the 
movable structure and thereby to the operator. The applied forces are based on internal programming within the programmable unit, events 
occurring in the simulation and movements of and forces applied to the movable structure by the operator. The system of the invention 
thereby provides continuous interaction among motion of and forces applied by the movable structure, the events in the simulation and 
forces applied to and applied by the movable structure. 
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INTERACTIVE SIMULATION SYSTEM INCLUDING FORCE FEEDBACK INPUT DEVICE 

Background of the Invention 

This invention relates to interactive simulations, and more particularly 
5 to simulations and games including force or touch feedback to a joystick or 
multi-axis controller or input device. 

Interactive computer simulations and interactive video games often 
include a movable structure generally known as a joystick to provide inputs 
for the simulation or game. Oftentimes, the joystick is strictly an input 
10 device by which signals are conveyed to the processing unit of the computer. 
These joysticks do not provide any tactile or force feedback to the user. 

The Atari Hard Drivin® video game provides one-dimensional force 
feedback to a steering wheel or a brake pedal. That is, during a game, 
torques are generated on the steering wheel which simulate those torques 
15 which would be experienced during, for example, cornering in a car. See 
also, U.S. Patent No. 5,044,956. In a more general simulation context, 
control elements including force feedback are known in elaborate flight 
simulators costing many millions of dollars. Heretofore, there has been no 
multiple-degree-of-freedom joystick suitable for games and simulations and 
20 suitable for arcade and home use. 

Summary of the Invention 

The interactive simulation system of the invention includes a 
computer or game display unit programmed with simulation rules. A 

25 control unit is connected to the computer and provides information and 
control signals to the computer to interact with the stored simulation rules 
and to receive information and control signals from the computer. A 
movable structure for movement in at least two degrees-of-freedom by an 
operator is interconnected to the control unit by means of actuators. A 

30 programmable unit processes signals and information from the computer to 
generate signals to drive the actuators so as to apply forces in the at least 
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two degrees-of-freedom to the movable structure and thereby to the operator. 
The applied forces are based on internal programming within the 
programmable unit, events occurring in the simulation and movements of 
and forces applied to the movable structure by the operator. The system 
5 thereby provides continuous interaction among motion of and forces applied 
by the movable structure, the events in the simulation and forces applied to 
and applied by the movable structure. 

In a preferred embodiment the simulation system is an interactive 
video game and the programmable unit resides in the control unit. In this 

10 embodiment, the movable structure is a joystick with two degrees-of- 
freedom. The actuators include electric motors driving a low-friction, low- 
backlash transmission. The programmable unit includes integrated 
electronics for controlling feedback and modeling user interaction with the 
simulation or game. It is also preferred that information compression 

15 technology be used so that much smaller information sets can be processed 
and communicated while still achieving a desired experience. 

By providing touch or force feedback, the system of the invention 
more effectively involves the operator in the simulation or game. For 
example, in two-dimensional games such as flight simulators the present 

20 touch feedback is used to allow the player to feel the difference in 
performance between a small general aviation airplane and a high 
performance military fighter and to feel environmental forces such as air 
turbulence or updrafts. In this case the joystick, known as a PowerStick™, 
will feel a variety of programmable effects including feeling "stiffer" or 

25 "softer" depending on which type of airplane is being "flown" by the 

simulation user. In three-dimensional games (which are just now becoming 
available) force feedback is particularly valuable because navigating in a 
three-dimensional environment is often difficult and the touch feedback 
provided by the invention will enable the user to feel objects which are 

30 being encountered and provide a real-time indication of where an object 
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controlled by the user or player is. In games involving contact, such as 
fighting games, the invention simulates and displays to the operator's hand 
the feel of collisions with objects encountered during the game or simulation 
and interaction with terrains of different textures. 



Brief Description of the Drawing 

Fig. 1 is a block diagram illustrating the overall system of the 
invention. 

10 Fig. 2 is a plan view, with parts broken away, of the force feedback 

control unit of the invention. 

Figs. 3 and 4 are side and front cross-sectional views, respectively, of 
the joystick control unit of the invention. 

Fig. 5 is a cross-sectional view of an alternative embodiment of the 
15 invention. 

Fig. 6 is a bottom view of the embodiment of Fig. 5. 
Fig. 7 is a detail taken from Fig.5. 
Fig. 8 is a detail taken from Fig. 6. 

Fig. 9 is a block diagram of the signal and communication paths in 
20 the operation of the invention. 

Figs. 10 and 11 are perspective illustrations of a character in an 
interactive game with force feedback according to the invention. 

Description of the Preferred Embodiment 

25 The overall system of the invention is illustrated schematically in Fig. 

1. A control unit 10 includes a movable structure such as a joystick 12. 
The control unit 10 also includes a programmable section 14. It will be 
appreciated that the unit 14 need not be a part of the control unit 10; it could 
be separate or part of the computer 16. A general purpose digital computer 

30 16 such as a personal computer or game display unit is programmed with 
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simulation or game rules. The output from the computer 16 is displayed on 
a video monitor 18. The control unit 10 is connected to the computer 16 
and provides information and control signals to the computer 16 to interact 
with the stored simulation rules and to receive information and control 
5 signals from the computer 16. The programmable unit 14 processes signals 
and information from the computer 16 and generates inputs to, and processes 
outputs from the joystick 12. 

With reference now to Fig. 2, the control unit 10 supports the 
movable structure or joystick 12 for two degrees-of-freedom motion with 

10 respect to the control unit 10. The joystick 12 also includes at least one 
button type switch 20 for sending signals to the computer 16. 

With reference now to Figs. 3 and 4, the control unit 10 is shown 
without its cover. The control unit 10 includes electric motors 30 and 32 
along with angular rotation sensors 34 and 36. The motors 30 and 32 

15 include shafts bearing capstans 38 and 40. The sensor 34 responds to the 
angular orientation of capstan 38 and the sensor 36 responds to the angular 
orientation of the capstan 40 to generate signals for use by the 
programmable unit 14. A cable transmission mechanism driven by the 
capstan 38 is adapted to rotate the joystick 12 about an axis 42, that is, left 

20 and right in Fig. 3. Similarly, the capstan 40 is adapted to rotate the 

joystick 12 about an axis 44 so that it moves into and out of the plane of 
Fig. 3. As shown in Fig. 4, a drum 46 cooperates with the capstan 38 to 
effect rotation. This transmission system has low backlash and low friction. 
The sensors 34 and 36 provide an output proportional to angular orientation 

25 of the respective motor 30 and 32 shafts. 

As shown in Fig. 1, the control unit 10 is connected to the computer 
16 so that information and control signals can pass in either direction 
between the control unit 10 and the computer 16. The programmable unit 
14 may be included in the control unit 10 housing or may be part of the 

30 computer 16 itself. The programmable unit 14 processes signals from the 
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computer 16 using internally stored rules and equations to generate signals to 
drive the motors 30 and 32 so as to produce forces with two degrees of 
freedom to the joystick 12 and to a user (not shown) operating the joystick 
12. The applied forces are based on internal programming within the 
5 programmable unit 14, events occurring in the simulation and movements of 
and forces applied to the joystick 12 by an operator (not shown). The 
system therefore provides continuous interaction among motion of the 
joystick 12, events in the simulation and the forces applied to and applied by 
the joystick 12. 

10 With reference now to Figs. 5, 6, 7 and 8, another embodiment of 

the control unit 10 is described. A motor 50 through a linkage denoted 
generally at 52 rotates a shaft 54. The shaft 54 passes through an angular 
orientation sensor such as potentiometer or encoder 56. The shaft 54 causes 
a bracket 58 to move left and right in Fig. 6 and engages a lower portion 59 

15 of the joystick 12 to cause the joystick 12 to move left and right in Fig. 5. 
The slotted bracket 58 is supported in bearings 60 and 61 . 

Similarly, a motor 62 drives a linkage 64 which rotates a shaft 68 
which passes through a potentiometer or encoder 70. The shaft 68 causes a 
bracket 72 to rotate as shown in Fig. 6 to cause the joystick 12 to move into 

20 and out of the plane of Fig. 5. The bracket 72 is supported by bearings 73 
and 74. 

As will be appreciated, as the motors 50 and 62 are energized, the 
joystick 12 is driven with two degrees-of-freedom, that is, left and right and 
in and out of the plane of Fig. 5. The rotational sensors 56 and 70 provide 
25 electrical outputs proportional to the angular orientation of their respective 
shafts. With reference to Fig. 5, the programmable unit 14 is shown as a 
circuit board 76 which may include the motor control and signal processing 
electronics. 

With reference now to Fig. 9 the programmable unit 14 includes a 
30 ■ microcontroller 80 which communicates with the computer 16 (Fig. 1) 
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through a host connection 82. The microcontroller 80 is any suitable 
processor such as a digital signal processor (DSP) or RISC architecture 
processor for enhanced operation. Further, the system can be designed for 
an upgradable processor core, independent of I/O hardware. In addition, the 
5 possibility exists of changing the controller's executable code from the host 
computer. This flexibility opens up the opportunity to customize the 
operation of the device for specific host applications. The microcontroller 
80 executes boot code upon start up that loads in default operating firmware 
from on-board non-volatile memory (or removable media, e.g., memory 

10 cards) and waits for a host command to begin execution, or to download 
operational software. The default operation code provides general force 
feedback, positional, and other functions. Code downloaded to the 
microcontroller 80 from the host computer 16 can be tuned for specific 
functions and/or characteristics. The downloaded code can either be native 

15 executable binary or pseudocode for an on-board interpreter. An interpreter 
implementation removes the need for the host to know what processor 
resides on the control unit. Amplifiers 84 and 86 drive the motors 30 and 
32 in the embodiment of Fig. 3 or the motors 50 and 62 in the embodiment 
of Fig. 6. It is to be noted that currents in the drive motors can be used to 

20 determine torque which can also be inferred from strain gauges or other 
types of force/torque sensors (not shown) on drive cables or shafts. It 
should also be noted that the torque transmission can be a four-bar linkage 
as illustrated in Figs. 5 and 6, a cable transmission illustrated in Figs. 3 or 4 
or a belt or gear train. If desired for a particular application, a speed 

25 reduction can be built into the transmission to amplify the torque applied to 
the joystick 12. In applications where space and motor size are not 
constrained, the motors 50 and 62 can be directly mounted onto shafts 54 
and 68 in Fig. 6 without transmissions. 

An example of a game or simulation illustrating the technology of the 
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invention disclosed herein will now be described in conjunction with Figs. 
10 and 11. The system of the invention has been integrated into an IBM 
Windows demonstration game which itself does not form a part of this 
invention. In this exemplary game, a character 90 fights his way through a 
5 series of different terrains to reach a treasure (not shown). The walking, 
running, jumping, squatting, punching and kicking of the character 90 are 
controlled by movements of the joystick 12. When, for example, the 
character 90 hits a patch of ice, the force being felt by the player goes to 
zero and can in fact use the motion and forces applied by the operator to 

10 further exaggerate the effect by actively getting out of the way of the 

operator to simulate sliding. On rough terrain, vibrations and pulses are felt 
by the game player in proportion to the objects encountered. If the 
character 90 is jumping and hits a tree or other object, the player will feel 
the stiffness of the object through forces transmitted through the joystick 12. 

15 When the button 20 is depressed, the character 90 stops moving and kicks or 
punches as shown in Fig. 11. Depending on the nature of the object being 
punched or kicked, and also depending on other events in the game, the 
forces experienced by the player will be different. Also, the velocity of the 
motions change the forces felt as they would in real fighting. Real physical 

20 models of the environment and objects encountered are programmed into the 
programmable unit 14 so that feedback is virtually instantaneous and 
independent of the graphics update of the computer 16. This architecture 
also makes the game fully interactive in which game outcomes unanticipated 
by the programmers occur based on the performance of the player. Because 

25 of the force feedback, the player is more actively involved in the game since 
the forces applied to the joystick 12 realistically simulate forces which would 
be felt by the character 50 in the game. 
What is claimed is: 
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Claims 

1. Interactive simulation system comprising: 
a computer programmed with simulation rules; 

a control unit connected to the computer and adapted to provide 
information and control signals to the computer to interact with the stored 
simulation rules and to receive information and control signals from the 
computer; 

a moveable structure adapted to be moved with at least two degrees- 
of-freedom by an operator and interconnected to the control unit through 
actuators; and 

programmable means for processing signals from the computer to 
generate signals to drive the actuators to apply force and touch feedback in 
the at least two degrees-of-freedom to the moveable structure and to the 
operator, the applied forces based on internal programming within the 
programmable means, events occurring in the simulation and movements of 
and forces applied to the moveable structure by the operator, whereby there 
is continuous interaction among motion of the moveable structure, the events 
in the simulation and forces applied to and applied by the moveable 
structure. 

2. The interactive simulation system of claim 1 wherein the 
programmable means is included with the control unit. 

3. The interactive simulation system wherein the actuators 
comprise: 

at least two motors for generating forces about respective axes of 
rotation. 
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4. The interactive simulation system of claim 1 wherein the 
actuators include at least one cable drive. 

5. The interactive simulation system of claim 1 wherein the 
actuators include a four-bar linkage. 

6. The interactive simulation system of claim 1 wherein the 
actuators include electrical motors and each motor includes means for 
determining angular orientation of the motor shaft. 

7. The interactive simulation system of claim 1 wherein the 
programmable means is located apart from the control unit. 

8. The interactive simulation system comprising: 
a computer programmed with simulation rules; 

a control unit connected to the computer and adapted to provide 
information and control signals to the computer to interact with the stored 
simulation rules and to receive information and control signals from the 
computer; 

a movable structure adapted to be gripped and moved by an operator 
in two degrees-of-freedom and interconnected to the control unit by means 
of electric motors and a transmission; and 

programmable means associated with the control unit for processing 
signals from the computer to generate signals to drive the motors to apply 
forces in the two degrees-of-freedom to the movable structure and to the 
operator, the applied forces based on internal programming within the 
programmable means, events occurring in the simulation and movements of 
and forces applied to the movable structure by the operator, whereby there is 
continuous interaction among motion of the movable structure, the events in 
the simulation and forces applied to and applied by the movable structure. 
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9. Control unit for interacting with a computer, comprising: 

a movable structure adapted to be moved with at least two degrees-of- 
freedom by an operator; 

actuators adapted to apply force and touch feedback to the movable 
structure with at least two degrees-of-freedom; and 

programmable circuitry for processing information to generate signals 
to drive the actuators to apply force and touch feedback in the at least two 
degrees-of-freedom to the movable structure and to an operator. 



PCT/US95/06204 




FIG. 2 



SUBSTITUTE SHEET (RULE 26) 



WO 95/32459 



PCT/US95/062W 



3/6 




FIG. 6 

SUBSTITUTE SHEET (RULE 26) 



WO 95/32459 



PCTAJS95/06204 



4/6 




FIG. 8 



SUBSTITUTE SHEET (RULE 26) 



WO 95/32459 



PCT/US95/06204 



5/6 



O 

o 



o 
o 

o 





Q£ 


00 

\ 


UJ 

1 

1 

o 


La- 




Of 






o 
o 




2 




MICF 



o 

00 





WO 95/32459 



PCT7US95/06204 



6/6 



90 




FIG. 11 
SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



Intent „nal Application No 

PCT/US 95/06204 



A . tXASSII 'K^IJON Ol SUIUIiCT MAlTf-K 

IPC 6 G06F3/00 G06K11/18 G09B9/28 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. I IIXDSSUARCIIlin 



Minimum documcntauon searched (dassificauon system followed by classification symbols) 

IPC 6 G06F G09B G06K 



Documentation .searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the inlcmauonal search (name of data base and, where practical, search terms used) 



C. DOCUMENTS CONSIDKRKD TO UK RKI,F.VANT 



Category * 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X 


US, A, 3 919 691 (A. MICHAEL NOLL) 11 

November 1975 

see the whole document 


1-9 


X 
Y 


EP,A,0 489 469 (ENGEL, FREDERICK LOUIS) 10 
June 1992 

see the whole document 


1-3,8,9 
4-7 



FR.A.l 102 320 (M. RICHARD CARL DEHMEL) 19 

October 1955 

see the whole document 



1-3,8,9 
4-7 



| | Further documents arc listed in the continuation of box C. 


|X | Patcnt family members arc listed in annex. 


*A" document defining the genera! stale of the art which is not 
considered to he of particular relevance 

"IT earlier document hut published on or after the international 
filing date 

"L" document which may throw doubts on priority daim{s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

*0* document referring to an oral disclosure, use, exhibition or 
other means 

*P" document published prior to the international filing date but 
later than the priority dale claimed 


T* later document published after the international filing date 
or prion ty date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot he considered to 
involve an inventive step when the document is taken alone 

*Y" document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

"A" document member of the same patent family 




Date of mailing of the international search report 


28 August 1995 


1 1. 09. 95 


Name and mailing address of the ISA 

Kuropcan Patent Of ficc, P.O. 5818 PatcnUaan 2 
NL - 2280 1 IV Rijswijk 
Tel. ( i 31.70) 340-2040, Tx. 31 651 cpo nl, 
Fax:( » 31-70) 340-3016 


Authorized officer 

Cervantes , J 





